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TEMPORAL IMAGE COMPARISON 

METHOD 

Background of Invention 

[0001] The present invention relates generally to imaging systems and more particularly 
to a method to improve the display of temporal changes. Imaging devices, such as x- 
ray machines, are widely used in both medical and industrial applications. Imaging 
devices often use temporal processing to track change in an object over time. 

[0002] Temporal processing systems typically include the following general modules: 

acquisition storage module, segmentation module, registration module, comparison 
module, and reporting module. The input Images are 1 -D, 2-D, 3-D, derived, 
synthesized, or montaged, where multiple separate images from a single time point 
are combined to provide a larger composite, seamless image. 

[0003] Detection of change in medical images of a patient, which are acquired at two 

different instances In time, has great potential for improving diagnosis. The advent of 
digital imaging allows computer-assisted detection and identification of these 
changes and the creation of a "dissimilarity image" containing the change information. 
This dissimilarity image can be read by a human controller or can become the input to 
an automated analysis device such as a CAD (computer assisted diagnosis) algorithm. 

[0004] Currently, as part of Mitsubishi Space Software's "temporal subtraction" 
application, dissimilarity images are calculated using a simple pixel-by-pixei 
subtraction of registered images. Simple subtraction, however, results in images with 
poor contrast, and is not substantially robust when the two initial images are acquired 
using different techniques. 

[0005] 

For a temporal subtraction image, the resulting pixel values (and hence the 
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displayed gray-levels) are proportional to the difference or dissimilarity in pixel value 
between two input images acquired with temporal separation. 

[0006] Input images are often registered and processed to compensate for several factors 
such as: the difference in positioning of the subject during the two image 
the difference in acquisition parameters, the difference in the bit resolution of the 
images, and the differences in any pre or post processing that may have been applied 
to the images. 

[0007] Image comparison is a common task In a number of applications including 

temporal processing of mono-modality Images (1-D, 2-D. 3-D, derived, synthesized, 
montaged). Current methods involve simple arithmetic operations conducted on these 
Images, including subtraction or addition, which are non-adaptive in terms of the 
spatial Image content. Therefore, there Is a need to improve the comparison module 
with more sophisticated methods, which are adaptive and which provide better results 
for subsequent processing and display techniques. 

[0008] The disadvantages associated with current, imaging systems have made it 

apparent that a new technique for temporal processing Is needed. The new technique 
should substantially increase accuracy of information acquired obtained from 
temporal processing. The present invention is directed to this end. 

Summary of Invention 

[0009] 

In accordance with one aspect of the present invention, a temporal image 
processing system includes a temporal processing controller adapted to receive a first 
image signal and a second image signal from a scanning unit. The temporal 
processing controller comprises a segmentation module adapted to isolate at least 
one region of interest of the first image signal and the second image signal. The 
segmentation module is adapted to generate therefrom a segmentation signal. The 
temporal processing controller further comprises a registration module adapted to 
receive the segmentation signal and register at least one region of interest. The 
registration module is further adapted to generate therefrom a registration signal. The 
temporal processing controller still further comprises a comparison module adapted 
to receive the segmentation signal and the registration signal. The comparison 
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module generates therefrom an adaptive comparison signal of the first image signal 
and the second image signal. 

[001 0] In accordance with another aspect of the present invention, a temporal image 
processing method is provided. The method includes scanning an object and 
generating a first image signal and a second image signal therefrom. The first image 
signal and the second image signal are received in a segmentation module. At least 
one region of interest of the first image signal and the second image signal is 
isolated. A segmentation signal is also generated and received in a registration 
module. The at least one region of interest is registered and a registration signal is 
generated. The segmentation signal and the registration signal are received in a 
comparison module, and an adaptive comparison signal is generated in response to 
the segmentation signal and the registration signal. 

[OOn] Additional advantages and features of the present invention will become apparent 
from the description that follows and may be realized by the instrumentalities and 
combinations particularly pointed out in the appended claims, taken in conjunction 
with the accompanying drawings. 

Brief Description of Drawings 

[001 2] For a more complete understanding of the invention, there will now be described 
some embodiments thereof, given by way of example, reference being made to the 
accompanying drawings, in which: 

[001 3] FIGURE 1 is a diagram of an imaging system in accordance with an embodiment of 
the present invention; 

[001 4] FIGURE 2 is a schematic diagram of a portion of FIGURE 1 ; 

[001 5] FIGURE 3 is a schematic diagram of the temporal processing unit of FIGURE 1 ; and 

[001 6] FIGURE 4 is a block diagram of a temporal processing method, in accordance with 
an embodiment of the present invention. 

Detailed Description 

[001 7] present invention is illustrated with respect to a temporal image processing 
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system 10, particularly suited to the medical field. The present invention is, however, 
applicable to various other uses that may require temporal imaging, as will be 
understood by one skilled in the art.Referring to FIGURES 1 and 2, an image controller 
12, including a temporal processing controller 14, coupled to a scanning unit 16, in 
accordance with an embodiment of the present invention, is illustrated. The scanning 
unit 16 includes, for example, an x-ray source 10 coupled to a gantry 20, generating 
an x-ray flux 22, which passes through an object 24 (e.g. a patient) on a table 1 5. The 
system 10 further includes an x-ray detector 26, also coupled to the gantry 20. 

[001 8] The imaging controller 1 2, including the temporal processing controller 14 and 
various other widely known imaging control and display components, receives the 
detector signals and responds by generating a first and a second Image signal. The 
imaging controller 1 2 also includes, for example, an operator console 28, a monitor 
and user interface 29, an x-ray controller 30, a table control 32, a gantry motor 
control 32, a mass storage 38. an image reconstructor 41 and a data acquisition 
system 42, all of which couple to a host computer and display 43, are well know in the 
art, and will be discussed later. 

[001 9] Referring to FIGURE 3, a schematic diagram of the temporal processing controller 
14 of FIGURE 1 is illustrated. One embodiment of the temporal imaging controller 
includes an acquisition storage module 40, a processing module 42, a segmentation 
module 44, a registration module 46, a comparison module 50, and a report module 
52. The aforementioned modules are either software algorithms or discrete circuit 
components or a combination thereof, as will be understood by one skilled in the art. 

[0020] Referring again to FIGURES 1 . 2 and 3, the acquisition storage module 40 contains 
acquired Images. For temporal change analysis, a data retrieval system retrieves the 
data from storage (e.g. the mass storage unit 38) corresponding to an earlier time 
point or from the image reconstructor 41 , which receives signals from the data 
acquisition system (DAS) 42. The acquisition storage module input images are 1-D, 2- 
D, 3-D, derived, synthesized, or montaged, such that multiple separate images from a 
single time point are combined to provide a larger composite, seamless image. 

[0021] an illustrative example, only two images SI and S2 corresponding to two time 

points t and t are included, however one skilled in the art will realize that the 
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general approach can be extended for any number of images in the acquisition and 

temporal sequence. The temporal comparison image is denoted as S ^ 2 * 

are either the original unprocessed images from an acquisition system or alternately, 

post-processed versions of the original images. One skilled in the art will realize that 

the acquisition storage module 40 can include images from almost any digital imaging 

source. 

[0022] The segmentation module 44 receives the two images, SI and S2 , from the 

acquisition storage module 40 and, through automated or manual operation in the 
user interface 29 or operator console 28, isolates regions of interest between the two 
x-ray images, SI and S2. Often the entire image is the region of interest, 

[0023] A segmentation mask (M) is generated from segmentation module logic, or 

alternately is input by a scan operator. The segmentation mask (M) is an application- 
dependent process for isolating at least one region of interest. For example, for 5 
areas of interest, each of the 5 areas receives an index number between 1 and 5, for 
the first image, and a corresponding designation is applied to each of the areas of 
interest for the second image. All other portions of the object are designated either 
"0" or in an alternate, known distinguishing manner (e.g. color indexing, etc.), as will 
be discussed later regarding the comparison module, 

[0024] The registration module 46 receives the region of interest signals from the 

segmentation module 44, provides methods of registration and therefrom generates a 
registration signal. If the regions of interest for temporal change analysis are small, 
rigid body registration transformations, including translation, rotation, magnification, 
and shearing, are sufficient to register a pair of images from t ^ and t ^ • If. however, 
the regions of interest are large, including almost the entire image, warped, elastic 
transformations are applied. 

[0025] One way to implement the warped registration is to use a multi-scale, multi- 
region, pyramidal approach. For this approach, a different cost function highlighting 
changes is optimized at every scale. Such cost functions are correlation methods, such 
as mathematical correlation and sign-change measurement, or statistical methods 
such as entropy measurements and mutual information analysis. 



APP ID=1 0064551 



Page 5 of 20 



[0026] For warped registration, images are re-sampled at a given scale and subsequently 
divided into multiple regions. Separate shift vectors are calculated for different 
regions. Shift vectors are interpolated to produce a smooth shift transformation, 
which is applied to warp one of the images. The images are re-sampled and the 
warped registration process is repeated at the next higher scale until the pre- 
determined final scale is reached. In other situations, a combination of rigid 
registration and elastic transformations is used. 

[0027] The comparison module 50 receives the segmentation signal and the registration 
signal and computes a dissimilarity measure between the registered images and 
therefrom generates a comparison signal. Registered image comparison is performed 
in several ways. In addition to, or instead of, using a simple subtraction between the 
registered images to obtain the dissimilarity image, the system includes an enhanced 
division method, which is described as (SI * S2)/(S2 * 52 + O ). 

[0028] For mono-modality temporal processing, the prior art methods obtain difference 
image D=S1S2. The present invention includes a method for adaptive image 
comparison between two images, SI and 52. An adaptive method includes the 
following equation: Dl ^ =(51 * 52)/(52 * 52 + O ), where the scalar constant O >0. 
For a degenerative case. <t) =0, the above equation becomes a straight division, 51 /52. 
A more sophisticated method includes O as a variable having values set according to a 
look-up table, as will be understood by one skilled in the art. In an embodiment, the 
input to the comparison module 50 also includes a segmentation mask (M), generated 
from the segmentation module 44 or from user input, in addition to 51 and 52. M is 
obtained from prior knowledge of the specific features in the compared images, such 
as bone structure or organ type. The segmentation mask guides the comparison 
results at various locations, as will be understood by one skilled in the art. 

[0029] For example, the following comparison method depending on spatial attributes: If 

M>0. Dl =(51 * 52)/(52 * 52 + O ); Else Dl =0. 
a a 

[0030] A general form of this method is the following: If M=ml , Dl =(S1 * S2)/(S2 * 52 

a 

+ 0 k); Else if M=m2, Dl =(S1 * S2)/(S2 * S2 + O k); Else if M=mk. Dl =(S1 * S2)/ 

a a 

(S2 * S2 + O k);Else Dl =0. 
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[0031] Where m is a variable designating which index is focused on in the referenced 
equation, and k is a constant, developed through known testing methods. 

[0032] The report nnodule 52 receives the comparison signal and provides the display and 
quantification capabilities for the user to visualize and or quantify the results of 
temporal comparison. Results of temporal comparisons are simultaneously displayed 
on a second display unit with either image SI or S2. Or a superposition of S ^ ^ onto 
SI or S2 generates with a logical operator based on pre-specified criterion. For 
quantitative comparison, color look-up tables for the overlaid images are used. The 
resulting combination is realized with a multi-color overlay display. 

[0033] The present invention is illustrated with respect to x-ray and computed 

tomography (CT) systems, however it is alternately used for any type of imaging 
system including, magnetic resonance imaging (MRI), mammography, vascular x-ray 
Imaging, bone scanning, positron emission tomography (PED), ultrasound, optical 
imaging, etc. Further embodiments include non-medical applications such as weld 
inspection, metal inspection. Essentially, anything that could use an imaging system 
to make one, two and three dimensional images or one, two, and three dimensional 
montaged images. 

[0034] Typical scanning units include an x-ray source 1 8 coupled to a gantry 20. The x~ 
ray source 1 8 generates an x-ray flux 22, which passes through a scanned object 26 
on a table 25. An x-ray detector 26 is also typically coupled to the gantry 20 such that 
the detector 26 receives the x-ray flux 22. 

[0035] The x-ray source 1 8 is embodied as a flat panel x-ray source or an extended x- 
ray source (e.g. Imatron), or a standard x-ray tube. The x-ray source 1 8 is activated 
by either a host computer 43 or an x-ray controller 30, as will be understood by one 
skilled in the art. The x-ray source 1 8 sends the x-ray flux 22 through an object 24 
on a moveable table 25 controlled by a table control device 37 acting in response to 
signals from the host computer 43, as will be understood by one skilled in the art. 

[0036] The embodied gantry 20 Is a ring shaped platform that rotates around the 

scanned object 24 in response to signals from the gantry motor control 34. as will be 
understood by one skilled in the art. 
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[0037] For a single image, SI , the x-ray flux 22 from the x-ray source 1 8 passes through 
the object 24 and impinges on the x-ray detector 26. Detector specific corrections or 
calibrations are then engaged, as will be understood by one skilled in the art. The 
signal then passes to the host computer and display 43, where the signal is converted 
to a gray level corresponding to the attenuation of an x-ray photon through the 
patient. The image is then stored in the mass storage unit 38 or received in the 
temporal imaging controller 14. 

[0038] The detector 26 is typically located opposite the x-ray source 1 8 to receive x-ray 
flux 22 generated therefrom. 

[0039] The host computer 43 receives detector signals. The host computer 43 also 

activates the x-ray source 1 8 through signals from the operator console 28 or user 
interface 29 however, alternate embodiments include independent activation means 
for the x-ray source 1 8. The present invention includes an operator console 28 for 
control by technicians, as will be understood by one skilled in the art. 

[0040] One embodiment of the present invention incorporates use of temporal imaging 
for the scout scan on an imaging system. During a scout scan from the x-ray source 
1 8 to the detector elements 26, the x-ray tube remains stationary while the patient 
table 25 translates under the x-ray flux 22. This results in a two-dimensional image 
ideal for qualitative information and for locating the desired position for scanning 
during further temporal imaging. 

[0041] Referring to FIGURE 4, a block diagram of a temporal Image processing method 98 
is illustrated. The logic starts in operation block 100 where the object is scanned at 
different periods of time to generate image signals. 

[0042] Operation block 101 then activates, and the temporal processing controller 

acquires the image signals through the acquisition storage module and generates an 
acquisition signal containing the image signals. 

[0043] In operation block 102, the segmentation module receives the acquisition signal 
and isolates regions of interest. In one embodiment, operation block 103 activates, 
and the segmentation module generates a segmentation mask, which operates to 
guide the control results. Alternately, the segmentation mask is input into the 
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temporal processing controller from an outside source, as was previously discussed. 

[0044] In block 104, the temporal processing controller registers the regions of interest. 

[0045] In operation block 108 the area of concern is analyzed. A minor region is analyzed 
with at least one of translation criteria, rotation criteria, magnification criteria, or 
shearing criteria, thereby generating a cost signal (i.e. cost function or a figure of 
merit of the cost function) in operation block 1 14. 

[0046] For a major area, at least one warped transformation criterion is used, thereby 

generating a cost signal (i.e. cost function or a figure of merit of the cost function) in 
operation block 1 14. 

[0047] Operation block 1 1 6 then activates, and the comparison module receives the 
segmentation signal, the registration signal and the segmentation mask. The 
comparison module generates therefrom an adaptive comparison signal in operation 
block 1 20. Following the adaptive comparison, the report module receives the 
comparison signal and provides the display and quantification capabilities for the user 
to visualize and or quantify the results of temporal comparison. 

[0048] In operation, the temporal image processing method includes scanning an object 
and generating a first image signal and a second image signal therefrom. A 
segmentation module receives the first image signal and the second image signal, 
isolates at least one region of interest and generates therefrom a segmentation signal. 
A registration module receives the segmentation signal, registers the region of 
interest, and generates a registration signal. A comparison module receives the 
segmentation signal and the registration signal and generates an adaptive comparison 
signal In response thereto. The comparison module also receives a segmentation 
mask from the segmentation module or from a user input, which guides the 
comparison results at various locations. 

[0049] Prom the foregoing, it can be seen that there has been brought to the art a new 
temporal image processing system 1 0. It is to be understood that the preceding 
description of the preferred embodiment is merely illustrative of some of the many 
specific embodiments that represent applications of the principles of the present 
Invention. Numerous and other arrangements would be evident to those skilled in the 



APP ID=1 0064551 



Page 9 of 20 



^.c 5 j5 s .7r?' t".? V t. ^"-^ fc . ^.'ff 

art without departing from the scope of the invention as defined by the following 
claims. 
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